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Abstract 

The  design  of  an  activity  recognition  and  monitoring  sys¬ 
tem  based  on  the  eWatch,  multi-sensor  platform  worn  on 
different  body  positions ,  is  presented  in  this  paper.  The  sys¬ 
tem  identifies  the  user's  activity  in  realtime  using  multiple 
sensors  and  records  the  classification  results  during  a  day. 
We  compare  multiple  time  domain  feature  sets  and  sam¬ 
pling  rates ,  and  analyze  the  tradeoff  between  recognition 
accuracy  and  computational  complexity.  The  classification 
accuracy  on  different  body  positions  used  for  wearing  elec¬ 
tronic  devices  was  evaluated. 

1.  Introduction 

The  primary  goal  of  this  paper  is  to  study  the  effective¬ 
ness  of  activity  classifiers  in  a  multi-sensor  system  as  we 
vary  the  wearing  positions  of  the  sensors.  The  eWatch  is 
used  as  a  multi  sensor  platform  for  wearable  context-aware 
computing.  Previous  feature  extraction  studies  examining 
accelerometer  data  have  shown  that  it  is  a  viable  input  for 
detecting  user  states  when  it  is  worn  on  the  wrist  [1].  Moti¬ 
vated  by  other  possible  sensor  platform  locations,  especially 
with  mobile  communication  devices  such  as  a  cell  phone  or 
PDA,  we  designed  a  study  to  investigate  the  dependency  of 
the  eWatch  classification  accuracy  on  given  different  body 
positions.  We  investigate  wearing  the  eWatch  in  the  follow¬ 
ing  locations:  the  belt,  shirt  pocket,  trouser  pocket,  back¬ 
pack,  and  necklace.  The  results  of  the  study  would  help  us 
decide  on  the  best  position  to  place  such  a  sensor  platform, 
and  understand  the  nature  of  the  trade-off  between  wearing 
position  and  classification  performance. 

In  [1],  the  authors  used  multiple  accelerometers  worn  on 
a  person’s  body  to  recognize  their  physical  activity.  Sensor 
data  from  multiple  body  positions  was  combined  for  classi¬ 
fying  the  activities. 

In  [2],  a  low  power  sensor  hardware  system  is  pre¬ 
sented,  including  accelerometer,  light  sensor,  microphone, 
and  wireless  communication.  Based  on  this  hardware,  a  de¬ 


Figure  1.  eWatch  sensing  platform 

sign  method  for  a  context  recognition  system  is  proposed. 
It  evaluates  multiple  feature  sets  and  makes  the  tradeoff  be¬ 
tween  power  consumption  and  recognition  accuracy.  A  sys¬ 
tem  that  classifies  household  activities  in  realtime  with  a 
focus  on  low  power  consumption  is  presented  in  [3]. 

In  [5],  a  system  using  an  armband  based  sensor  array 
and  unsupervised  machine  learning  algorithms  was  able  to 
determine  a  meaningful  user  context  model. 

In  Section  2,  we  describe  the  sensor  platform  and  Sec¬ 
tion  3  explains  the  experimental  design.  Section  4  describes 
the  activity  recognition  method.  Section  5  presents  the  re¬ 
sults  of  the  data  analysis,  and  Section  6  addresses  the  per¬ 
formance  of  our  on-board  activity  classifier. 

2.  Sensing  Platform 

Our  sensor  platform,  the  eWatch  (Figure  1),  is  based  on 
the  Philips  LPC2106  ARM7  TDMI  microcontroller,  with 
128kB  of  internal  flash  memory  and  64kB  of  RAM  [6]. 
The  LPC2106  is  a  32bit  processor  running  at  up  to  60Mhz. 
eWatch  contains  four  sensors:  a  dual  axes  accelerometer, 
light,  temperature  sensor  and  microphone.  Sensor  data  can 
be  stored  in  a  1MB  external  flash  memory. 

3.  Experiment  Design 

In  our  study  we  focussed  on  six  primary  activities:  sit¬ 
ting,  standing,  walking,  ascending  stairs,  descending  stairs 
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and  running.  Body  positions  that  are  normally  used  for 
wearing  electronic  devices,  such  as  cell  phones  or  PDAs, 
were  studied.  We  placed  our  sensor  hardware  on  the 
left  wrist,  belt,  necklace,  in  the  right  trouser  pocket,  shirt 
pocket,  and  bag.  The  subjects  wore  six  eWatch  devices  lo¬ 
cated  at  these  body  positions  during  the  study.  The  devices 
recorded  sensor  data  from  the  accelerometer  and  light  sen¬ 
sor  into  their  flash  memory.  The  user  was  asked  to  perform 
tasks  that  consist  of  the  activities,  such  as  working  on  the 
computer  or  walking  to  another  building.  The  lead  experi¬ 
menter  annotated  the  current  activity  and  instructed  the  sub¬ 
jects  on  how  to  proceed.  The  annotations  were  done  using 
an  application  running  on  an  extra  eWatch  worn  by  the  lead 
experimenter. 

Six  subjects  participated  in  the  study,  each  subject  per¬ 
formed  the  given  tasks  in  45  to  50  minutes.  In  total  we 
collected  over  290  minutes  of  sensor  data. 

Sensor  setup  eWatch  recorded  both  axes  of  the  ac¬ 
celerometer  and  the  light  sensor.  All  sensors  values  were 
recorded  with  a  frequency  of  50Hz  and  with  8bit  resolution. 
The  accelerometer  was  calibrated  so  that  both  axes  operate 
in  a  range  of  ±2g.  Evaluation  of  the  recorded  data  was  done 
with  Matlab  and  the  WEKA  software  [7]. 

4.  Activity  recognition  method 

The  sensor  values  recorded  from  the  accelerometers  and 
the  light  sensor  are  split  into  short  time  windows.  These 
windows  are  then  transformed  into  the  feature  space  by  cal¬ 
culating  several  feature  functions  over  the  individual  win¬ 
dows. 

Features  Features  from  both  accelerometer  axes  (X  &  Y), 
the  light  sensor,  and  a  combined  value  of  both  accelerom¬ 
eter  signals  were  calculated.  To  reduce  the  dependency  on 
the  orientation,  both  X  and  Y  values  were  combined  calcu¬ 
lating  the  squared  length  of  the  acceleration  vector.  The 
classification  accuracy  with  individual  sensors  as  well  as 
with  multiple  sensor  combined  was  investigated. 

Only  time  domain  features  were  considered  to  avoid  the 
costly  computation  that  is  required  to  transform  the  signal 
into  the  frequency  domain.  Table  1  shows  the  list  of  fea¬ 
tures  that  were  considered.  The  functions  to  calculate  these 
features  were  implemented  on  the  eWatch  and  the  required 
number  of  clock  cycles  per  function  was  measured.  Each 
function  was  executed  2000  times  with  different  recorded 
sensor  inputs,  and  then  the  average  value  was  computed. 
The  execution  time  was  calculated  based  on  the  measured 
clock  cycles  and  the  CPU  frequency  at  59MHz.  Table  1 
shows  the  measured  clock  cycles  and  execution  time  using 
a  four  second  window  sampled  at  20Hz  (80  samples). 

Figure  2  depicts  the  feature  space  after  a  transformation 
with  Linear  Discriminant  Analysis  (LDA).  It  shows  that  the 


Figure  2.  Feature  space  after  LDA  transfor¬ 
mation 


standing ,  sitting  and  running  activities  form  seperate  clus¬ 
ters,  while  walking ,  ascending  and  descending  stairs  are 
closer  together  since  these  activities  are  very  similar. 

Feature  Subsets  To  reduce  the  time  and  energy  required 
to  calculate  the  feature  vector,  several  subsets  of  the  com¬ 
plete  feature  space  were  evaluated.  Some  features  are  irrele¬ 
vant  or  redundant  and  do  not  provide  information  to  signifi¬ 
cantly  improve  the  classification  accuracy.  Therefore  a  sub¬ 
set  of  the  available  features  can  be  selected  to  decrease  the 
computation  time  without  significantly  decreasing  recogni¬ 
tion  accuracy. 

The  Correlation  based  Feature  Selection  (CFS)  method 
from  the  WEKA  toolkit  was  used  to  find  feature  sets  con- 


Features  /  Function 

Name 

Avg.  CPU 
Cycles 

Avg.  Time 
in  us 

Empirical  Mean 

mean 

854 

14.5 

Root  Mean  Square 

rms 

1219 

20.7 

Standard  Deviation 

std 

1139 

19.3 

Variance 

var 

1313 

22.3 

Mean  Absolute  Deviation 

mad 

1089 

18.5 

Cumulative  Histogram  (256  bins) 

hist 

5847 

99.1 

n’th  Percentile  (n  =  5, 10, .  . .  ,  95) 

prc 

142 

2.4 

Interquartile  Range 

iqr 

289 

4.9 

Zero  Crossing  Rate 

zcr 

993 

16.8 

Mean  Crossing  Rate 

mcr 

996 

16.9 

Sq.  Length  of  X,Y  (xz  +  y2) 

1318 

22.3 

Decision  Tree  classifier  (18  nodes) 

138 

2.3 

Table  1.  List  of  time  domain  features  and  the 
average  clock  cycles  and  time  to  calculate 
them  on  the  eWatch  running  at  59MHz 
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(a)  Features  from  the  accelerometer’s  (b)  Features  from  the  accelerome-  (c)  Features  from  the  light  sensor 
X-axis  ter’s  Y-Axis 


(d)  Features  from  the  x2  +  y2  value 
of  the  accelerometers 


Figure  3.  Recognition  accuracy  with  different  feature  sets 


taining  features  that  are  highly  correlated  within  the  particu¬ 
lar  class  but  are  uncorrelated  with  each  other.  Table  2  shows 
the  feature  sets  that  were  compared. 


Classificaton  method  We  evaluated  and  compared  sev¬ 
eral  classification  methods,  namely  Decision  Trees  (C4.5  al¬ 
gorithm),  k-Nearest  Neighbor  (k-NN),  Naive-Bayes  and  the 
Bayes  Net  classifier.  Decision  Trees  and  Naive-Bayes  were 
found  to  achieve  high  recognition  accuracy  with  acceptable 
computational  complexity.  Decision  Trees  were  used  for 
activity  classification  in  [1]  and  [2].  It  was  shown  in  [4]  that 
the  discretized  version  of  Naive-Bayes  can  outperform  the 
Decision  Tree  classifier  for  general  classification  problems. 
Finally  the  Decision  Tree  classifier  was  chosen  as  it  pro¬ 
vides  a  good  balance  between  accuracy  and  computational 
complexity.  For  all  further  experiments  this  classifier  with 
a  5 -fold  cross  validation  was  used. 


Sampling  frequency  During  the  user  study  the  sensors 
were  sampled  with  a  frequency  of  50Hz  and  later  down- 
sampled  to  lower  frequencies.  To  maintain  some  of  the  high 
frequency  components  information  and  to  reduce  the  com¬ 
putational  complexity  significantly,  no  low  pass  filter  was 
used  for  downsampling  the  data.  Figure  3  shows  the  recog¬ 
nition  accuracy  for  different  sample  rates  from  1  to  30Hz 
for  the  different  body  positions.  The  recognition  accuracy 
was  defined  as  the  percentage  of  correctly  classified  feature 
vectors  averaged  for  all  six  activities.  The  recognition  ac¬ 
curacy  increases  with  higher  sampling  rates,  and  with  the 
accelerometer  features  the  accuracy  then  stabilizes  between 
15  to  20Hz,  and  is  only  improved  marginally  with  higher 
sampling  rates.  The  accuracy  with  the  light  sensor  only  is 
lower  and  it  stabilizes  beginning  with  7Hz.  In  Figure  3(c) 
the  results  from  the  belt  and  pocket  position  is  not  shown 
because  the  light  sensor  did  not  provide  any  useful  classifi¬ 
cation  information  at  these  positions. 


Figure  4.  Recognition  accuracy  for  the  activi¬ 
ties  at  different  body  locations 


5.  Data  Analysis  -  eWatch  Classification  Per¬ 
formance  for  Various  Wearing  Positions 

We  calculated  the  classification  accuracy  for  every  ac¬ 
tivity  on  each  of  the  six  different  body  positions.  The  data 
from  all  subjects  was  combined  to  train  a  general  classifier 
that  is  not  specific  to  a  person. 

Table  2  shows  the  feature  sets  and  the  classification  re¬ 
sults  for  the  different  body  positions.  The  features  are  cal¬ 
culated  from  a  20Hz  signal. 

Figure  4  shows  the  recognition  accuracy  for  the  individ¬ 
ual  activities  at  different  body  locations.  For  the  classifica¬ 
tion  the  reduced  feature  set  F6  was  used.  The  data  indicate 
that  any  of  the  six  positions  are  good  for  detecting  walking, 
standing,  sitting  and  running.  Ascending  and  descending 
the  stairs  is  difficult  to  distinguish  from  walking  in  all  posi¬ 
tions,  since  the  classifier  was  trained  for  multiple  persons. 
The  wrist  performs  best  because  the  feature  set  was  opti¬ 
mized  for  the  wrist  position. 

6.  Onboard  Activity  Classifier 

Based  on  these  results  we  implemented  a  decision  tree 
classifier  that  runs  on  the  eWatch.  The  feature  set  F6  was 
used  to  build  the  decision  tree.  The  sensor  sampling  is 
interrupt-based,  and  triggers  the  sampling  of  the  sensors  at 
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# 

Features 

CPU  Cycles 

Time  in  /zs 

Classification  Accuracy  for  Body  Position 

wrist 

pocket 

bag 

necklace 

shirt 

belt 

FI 

All  features,  all  sensors 

56242 

953.6 

87.1% 

85.2% 

92.8% 

86.8% 

89.5% 

87.0% 

F2 

All  features  from  accelerometer  X 

14731 

249.8 

68.4% 

78.6% 

85.2% 

74.3% 

76.1% 

72.6% 

F3 

All  features  from  accelerometer  Y 

14731 

249.8 

83.2% 

75.4% 

86.4% 

61.3% 

70.1% 

70.8% 

F4 

All  features  from  light 

14731 

249.8 

55.0% 

16.7% 

18.0% 

61.7% 

52.2% 

48.8% 

F5 

All  features  from  accelerometer  XY  ( xz  Ay2) 

15049 

255.2 

76.6% 

79.5% 

87.2% 

72.6% 

78.0% 

77.2% 

F6 

prey  (3) ,  rmsxy ,  prcy  (20) ,  prcy  (97) , 
rmsught,  madx,  meany,prcy  (10) 

12114 

205.4 

87.0% 

80.1% 

86.5% 

78.6% 

79.6% 

84.2% 

F8 

prey  (3) ,  iqry ,  prcy  (10),  prcy  (97) ,  madx 

7651 

129.7 

82.0% 

62.4% 

68.9% 

56.6% 

69.8% 

71.7% 

F9 

rmsxy ,  qrtx ,  rmsx ,  madxy ,  meanxy 

10746 

182.2 

77.3% 

78.2% 

80.9% 

72.3% 

75.4% 

76.5% 

Table  2.  Feature  sub  sets  and  classification  accuracy  for  body  positions 


Figure  5.  Activity  classification  recorded  over 
100  minutes 


20Hz.  The  sensor  value  is  stored  in  a  buffer  with  the  size  of 
the  sliding  window.  The  activity  is  classified  every  0.5  sec¬ 
onds  based  on  the  sensor  data  from  the  4  second  buffer.  The 
classification  results  are  stored  into  flash  memory  and  are 
downloaded  to  a  computer  later  for  further  processing  and 
analysis.  They  can  also  be  transferred  in  realtime  over  the 
Bluetooth  connection.  In  order  to  save  energy,  the  system 
remains  idle  between  servicing  interrupts. 

A  subject  wore  the  eWatch  with  the  built-in  activity  clas¬ 
sifier  on  the  wrist  during  the  day.  The  system  classified  the 
activity  in  realtime  and  recorded  the  classification  results 
to  flash  memory.  Figure  5  shows  100  minutes  of  activity 
classification,  as  the  user  walked  to  a  restaurant,  sat  down, 
ate  lunch,  went  back  to  the  office  and  sat  down  to  con¬ 
tinue  working.  The  classification  results  match  well  with 
the  actual  activities;  eating  lunch  was  partially  interpreted 
as  walking  or  running  activity  due  to  arm  movements. 

7.  Conclusions  and  Future  Work 

The  activity  recognition  and  monitoring  system  that  can 
identify  and  record  the  user’s  activity  in  realtime  using  mul¬ 
tiple  sensors  is  presented.  We  compared  multiple  feature 
sets  and  sampling  rates  to  find  an  optimized  classification 
method,  and  showed  how  well  they  perform  on  different 
body  locations  that  are  commonly  used  for  wearing  elec¬ 


tronic  devices. 

We  will  extend  our  activity  classifier  to  other  activites 
and  investigate  how  the  activity  classification  can  support 
the  recognition  of  the  user’s  location. 
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